If you want an incident to be created and automatically resolved, you can add a dummy user to your account, and then create a schedule for the dummy to be on-call during your off hours. From there, when an incident is triggered, PagerDuty will assign it to the dummy user. Then, as long as your service has an auto-resolve timeout setting, the incident will automatically be resolved.
It’s important to note that you will want to create schedules for the other users during the office hours, and have those schedules on the escalation policy.